January 14, 2002 

INFORMATION DISCLOSURE STATEMENT letter 
To Whom It May Concern: 

The following documents are included in form PTO/SB/08A Information Disclosure ^ 
Statement by Applicant as a bibliography for the application having the following title and g S 
applicant. 

Title: Digital Images with Composite Exposure 

Applicant: Charles D. Murphy S*-* 

Here, the documents are discussed as they relate to the specification and claims of the 
application. 



The present invention proposes a digital imaging technique in which members of a set of 
digital images of a field of view are arithmetically combined to form a composite digital 
image of the field of view. At least one of the digital images includes a pixel which 
represents the output of a sensor that responds to visible light. The composite digital 
image can have different exposure properties than the digital images in the set. For 
instance, pixels in the composite digital image can have differing exposure times than 
their counterparts in the digital images. Also, the invention can be used to compensate 
for or enhance effects such as motion blur and flash illumination. A key feature of the 
invention is the ability to provide desired exposure properties for interesting features in 
the field of view after the digital images have been acquired. This enables novice 
image-takers to make good composite digital images. Different viewers may produce 
different composite digital images on the basis of different features of interest as long as 
the information contained in the digital images has not been discarded. 



Many prior art techniques for digital imaging make use of image combination. 

A well-established technique in both analog and digital imaging is photomontage. In 
photomontage, an image of one field of view is combined with an image of another field 
of view to create a third image of a field of view that never existed. For instance, one 
might create an image of a city skyline on the night of a full moon, with the full moon 
replaced by an image of the planet earth taken from space. As another example, one 
image of a field of view containing a person might be combined with another image g 
containing the same person, making it appear that the person was in two locations s 
simultaneously. g S ^ 

i 

Photomontage is mainly intended as a form of art. The composite image may sugge: 
various ideas to viewers on the basis of the various fields of view and their features ti 
appear in the composite image. The effect of combining two images may be to starti 
the viewer, as in the examples given above. Alternatively, the viewer may not even 
realize that the field of view of the combined image was not an actual field of view for ^ 
which an image was acquired. g 
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The main difference between photomontage and the present invention is that 
photomontage creates an artificial field of view, whereas the present invention attempts 
to combine different digital images of an actual field of view to create a composite digital 
image of that actual field of view. There are embodiments of the present invention that 
compensate for or enhance the effects of motion of features of a field of view. Their 
purpose is not to create an artificial field of view, but to produce an image that is either a 
near-instantaneous representation of an actual field of view, or a representation of an 
actual field of view in which there is motion and in which it is desired that the motion be 
imaged. In the case of compensation, there was an actual time period when the imaged 
objects occupied the near-instantaneous positions. In the case of enhancement, there 
was an actual time period when the objects moved through a trajectory between a 
starting location and an ending location. Both enhancement of and compensation for 
motion in the present invention exploit the fact that information about relative positions or 
sets of relative positions is preserved in the digital images, whereas in a single image of 
longer duration the information is lost. 

It is possible to take a first digital image of a field of view and modify it by photomontage 
to produce a first modified digital image of an artificial field of view. Subsequently, the 
first modified digital image could be combined with a second digital image of the field of 
view. It is desired that a trivial modification which does not serve a useful purpose other 
than to avoid the language of the claims of the present invention, but which has 
substantially the same intended results as the present invention, be considered covered 
by the claims. 

In other words, the term "field of view" should be interpreted as referring to a set of 
features that during acquisition of an image are located in particular relative locations, or, 
in the case of motion, that are located in particular sets of relative locations. If, during 
acquisition of a second image, substantially the same set of features are located in 
substantially the same relative locations or sets of relative locations, the second image is . 
of the same field of view as the first image. If the second image of the same field of view • 
is modified, but still comprises substantially the same set of features located in 
substantially the same relative positions, the modified image should be considered still to 
be of the same field of view. 



Digital image acquisition techniques using sensor arrays typically produce large 
numbers of analog sensor outputs that must be converted to digital number values. The 
conversion usually takes place either one sensor output at a time or with small groups of 
sensor outputs converted simultaneously. The acquired digital image is simply the set of 
digital number values produced by the conversion process. However, the set of all 
digital number values produced by the conversion process with one number removed is 
clearly not identical. For instance, an acquired digital image less the digital number 
value of an arbitrary pixel is not identical to the acquired digital image. 

As another example, a color digital image is usually composed of triplets of digital 
number values. Each pixel may have a red value, a blue value, and a green value, for 
instance. Thus a color digital image can be considered as composed of a red digital 
image, a blue digital image, and a green digital image. 
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The progressive or interlaced scan techniques used to process analog sensor outputs 
into digital number values result in the building of a set of digital number values that 
compose a digital image. Likewise, color digital images are sets of digital number value 
triplets. 

With regard to the present invention, combining of digital number values by creating sets 
is not the same as arithmetic combining. In arithmetic combining, digital number values 
are modified using basic operations such as addition, multiplication, subtraction, 
truncation, rounding, or other operations. Thus, the present invention should not have 
any conflicts with prior art digital image scanning techniques or multi-spectral digital 
images. 



Prior art methods for radar imaging and sonar imaging are numerous. In most radar and 
sonar applications, there is an active source of illumination, usually of short duration. 
The imaging systems emit a pulse, which propagates from the source, reflects from 
some features in the field of view, and returns to a sensor array. The pulse may be short 
so that the same device can emit it and sense the return signal. However, a more 
important reason for having a short pulse is that the distance to various features in the 
field of view can be determined on the basis of the round-trip travel time of the pulse. 

Most radar and sonar imaging systems produce a time series of sensor outputs. There 
are often a small number of sensors whose outputs are measured at a large number of 
time epochs. A digital image of the field of view is formed by combining the digitized 
outputs from the various sensors at the various times. It is certainly possible to consider 
the digitized outputs of the sensor array at one sampling time as a digital image, which is 
arithmetically combined with the digitized outputs of the sensor array at other sampling 
times to produce a final digital image. This interpretation is quite similar to the core idea 
of the present invention, and is a reason for limiting the present invention by requiring a 
digital image in the first set with a digital number value representing the output of a 
sensor that responds to visible light. 

The aim of the present invention is exposure adjustment in digital imaging with visible 
light, where one digital image is usually a set of digital numbers representing 
simultaneous measurements of a large array of sensors. An individual regarding such a 
digital image can usually identify whether it is overexposed, underexposed, or properly 
exposed. Moreover, the individual can also discern the exposure properties of features 
that interest him or her. The present invention is intended to enable easy image 
acquisition and exposure adjustment by people unskilled in the art of digital imaging. On 
the other hand, a radar or a sonar image composed of one set of sensor measurements 
at one time instant generally is not easily interpreted by either a layperson or an expert. 
The digital images from the combined sensor outputs at the various sampling times is 
required to provide immediately useful information and an expert is needed for 
interpretation. 



US Patent 6,330,302 "Method of and device for forming an image of an object from a 
plurality of images", issued to J.H.M. Joosten on December 11, 2001 and US Patent 
5,436.952 "Telerecording system for x-ray images having a semiconductor image 
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converter" issued to J. Haendle and H. Horbaschek on July 25, 1995 represent prior art 
material from the area of medical x-ray imaging. Specifically, the patents deal with x-ray 
imaging In which there is a source of x-rays and an x-ray sensor or sensor array. 

X-rays are not visible light, and the present invention requires that one of the digital 
images of the field of view include a digital number value representing a sensor that 
responds to visible light. The goal of medical x-ray imaging is to provide images of 
internal body parts without any kind of intrusive surgery. Most people are familiar with x- 
ray images as a way of identifying broken bones. The x-rays pass through the human 
body, and attenuation of the x-rays provides variation in acquired images. 

It has been recognized for a long time that there are basically two types of tissue in the 
human body. Hard tissue, namely, bone tissue, absorbs x-rays at a particular rate for a 
given x-ray energy. Soft tissue such as flesh, on the other hand, absorbs x-rays at a 
much different rate. An x-ray energy or duration that produces desired sensor response 
variation for a hard tissue feature may produce little useful variation for a soft tissue 
feature. An x-ray energy or duration that produces desired sensor response variation for 
a soft tissue feature may produce little useful variation of a hard tissue feature. Thus, 
one type of x-ray is useful for imaging hard tissue, and another type of x-ray is useful for 
imaging soft tissue. 

The prior art of US Patents 6,330,302 and 5,436,952 proposes combining two or more x- 
ray images in order to produce a composite image in which hard tissue features and soft 
tissue features both have desired variation. The combining can be arithmetic combining. 



Another area of prior art is image joining. For instance. US Patent 5,982,951 "Apparatus 
and method for combining a plurality of image" issued to T. Katayama, H. Takiguchi, K. 
Yanoa, and K. Hatori on November 9, 1999 describes how to create a panoramic image 
of a large field of view from smaller images of smaller, different fields of view. 

The main features of image joining techniques are, first, identification of overiapping 
regions and establishing of proper relative locations for the images being joined, and 
second, modifying digital number values so that there are no joining artifacts such as an 
obvious boundary line of the overiapping region in the joined image. Image joining 
techniques can use arithmetic combining, for instance, to gradually shift contributions 
from one image to the other across the region of overiap. 

Image joining techniques differ from the present invention in several respects. Image 
joining techniques use images with different fields of view, whereas the present invention 
uses digital images with a single field of view. The different fields of view in image 
joining have small regions of overiap. The fields of view in the present invention are 
substantially entirely overiapping. Also, the goal of image joining is to create composite 
digital image of an artificial field of view. That is, the panoramic field of view never 
existed as a set of features all contributing to a single acquired image. In contrast, the 
present invention is intended to improve the exposure properties a composite digital 
image of an actual field of view via arithmetic combination of several digital images of 
the actual field of view. 
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I, the inventor of the present invention, have been unable to find any prior art closely 
similar to the material of the present invention other than the prior art discussed above. 
To the best of my knowledge, the idea of using several digital images of a field of view to 
produce a composite digital image of that field of view is new and deserving of patent 
protection. 

This concludes this INFORMATION DISCLOSURE STATEMENT letter. 



Sincerely, 




Charles D. Murphy 

1816 West Wilson Avenue #3 

Chicago, IL 60640 



INFORMATION DISCLOSURE STATEMENT LETTER 



5 



Please type a plus sign (+) Inside this box 



PTO/SB/08A (08-00) 
Approved for use through 10/31/2002. 0MB 0651-0031 

U.S. Patent and Trademark Office: U.S. DEPARTMENT OF COMMERCE 
Under the Paperwork Reduction Act of 1995. no persons are required to respond to a collection of information unless it contains a valid QMB control number. 



Substitute for form 1449A/PTO 



INFORMATION DISCLOSURE 
STATEMENT BY APPLICANT 

(use as many sheets as necessary) 



Complete if Known 



Application Number 



Filing Date 



First Named Inventor 



Group Art Unit 



Examiner Name 



Sheet 



Of 



I 



Attorney Docket Numt)er 



U.S. PATENT DOCUMENTS 


Examiner 
Initials* 


Cite 
No.^ 


U.S. Patent Document 


Name of Patentee or Applicant 
of Cited Document 


Date of Publication of 

Cited Document 
MM-DD-YYYY 


Pages, Columns. Lines, 
Where Relevant 

Passages or Relevant 
Figures Appear 


Kind Code^ 
Number . . 

[if known) 


































































































\ 

























































































































































































la 



FOREIGN PATENT DOCUMENTS 


Examiner 
Initials* 


Cite 

No."" 


Foreign Patent Document 


Name of Patentee or 
Applicant of Cited Document 


Date of Publication of 
Cited Document 
MM-DD-YYYY 


Pages. Columns, Lines, 

Where Relevant 
Passages or Relevant 




Kind Code* 

Office^ Numt)er {if known) 























































































































































































Examiner 




Date 




Signature 




Considered 





'EXAMINER: Initial if reference considered, whether or not citation is in conformance with MPEP 609. Draw line through citation If not in conformance and not 
considered. Include copy of this fonm with next communication to applicant 



1 Unique citation designation number. ^ see attached Kinds of U.S. Patent Documents. ^ Enter Office that issued the document, by the two-letter 
code (WlPO Standard ST.3). * For Japanese patent documents, the indication of the year of the reign of the Emperor must precede the serial 
number of the patent document. ^Kind of document by the appropriate symbols as Indicated on the document under WlPO Standard ST. 16 rf 
possible. ^Applicant is to place a check mark here if English language Translation is attached. 

Burden Hour Statement: This form is estimated to take 2.0 hours to complete. Time will vary depending upon the needs of the individual case. Any comments on 
the amount of time you are required to complete this form shoukJ be sent to the Chief Information Officer, U. S. Patent and Trademark Office, Washington, DC 20231. 
DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Assistant Commissioner for Patents, Washington, DC 20231 . 



